Systems and methods for switching vision correction graphical outputs on a display of an electronic device

ABSTRACT

A method of providing a graphical output may include scanning at least a portion of a user&#39;s face using a sensor; generating a depth map using the scan; and determining a similarity score between the depth map and a set of stored biometric identity maps that are associated with a registered user. In response to the similarity score exceeding a threshold, the user may be authenticated as the registered user. The method may further determine a corrective eyewear scenario, select a display profile that is associated with the corrective eyewear scenario, and generate a graphical output in accordance with the selected display profile.

TECHNICAL FIELD

Embodiments described herein relate generally to providing personalizedgraphical outputs and, in particular, to systems, processes, and methodsfor displaying vision-corrected graphical outputs and standard graphicaloutputs on an electronic device.

BACKGROUND

Modern electronic devices, such as mobile phones, smart phones, laptopcomputers, desktop computers, media players, gaming devices, and thelike, commonly include electronic displays which may provide a user withvisual information.

A large percentage of the human population requires prescriptioneyeglasses or contact lenses in order to see with sufficient clarity.For example, a person with nearsighted vision (myopia) may havedifficulty perceiving far away objects. Similarly, a person withfarsighted vision (hyperopia) may have difficulty perceiving nearbyobjects. In order to view an electronic display, a person with a visiondeficiency may need to put on or remove prescription eyewear to avoideye strain and/or to view the electronic display clearly. If such aperson is unable to easily remove or put on the prescription eyewear, itmay be difficult to interact with the electronic display and a userexperience with the electronic display may suffer.

SUMMARY

A method of controlling a vision-correcting operation of a portableelectronic device may include scanning at least a portion of a face of auser using a sensor, generating a depth map using the scan conductedusing the sensor, and determining a similarity score between the depthmap and one or more identity maps of a set of stored biometric identitymaps that are associated with a registered user. In response to thesimilarity score exceeding a threshold, the method may further includeauthenticating the user as the registered user and determining acorrective eyewear scenario using the depth map. The method may furthercomprise selecting a display profile that is associated with thecorrective eyewear scenario and the registered user and generating agraphical output in accordance with the selected display profile. Thecorrective eyewear scenario may correspond to the registered userwearing a corrective eyewear. The graphical output may compensate for avision deficiency associated with the corrective eyewear scenario andthe registered user.

The depth map may be a first depth map, the display profile may be afirst display profile, the corrective eyewear scenario may be a firstcorrective eyewear scenario, and the graphical output may be a firstgraphical output. The method of controlling a vision-correctingoperation may further comprise scanning at least the portion of the faceof the user using the sensor to generate a second depth map anddetermining a second corrective eyewear scenario using the second depthmap. The method may further comprise selecting a display profile that isassociated with the second corrective eyewear scenario and generating asecond graphical output in accordance with the selected second displayprofile. The second corrective eyewear scenario may correspond to theregistered user not wearing corrective eyewear.

The threshold may be a first threshold and the similarity score may be afirst similarity score. Determining the corrective eyewear scenariousing the depth map may comprise identifying a subset of identity mapsof the set of stored biometric identity maps, the subset of identitymaps associated with the corrective eyewear scenario, and determining asecond similarity score between the depth map and the subset of identitymaps.

The corrective eyewear scenario may correspond to the registered usernot wearing a corrective eyewear. The graphical output may compensatefor a vision deficiency while the user is not wearing the correctiveeyewear.

The method of controlling a vision-correcting operation may furthercomprise detecting an eye movement of the user and, in accordance withthe eye movement corresponding to an eye strain condition, modifying thegraphical output of the portable electronic device.

The display profile may be associated with prescription informationrelated to a visual acuity of the user and the graphical output may begenerated, at least in part, using the prescription information.

A method of providing a graphical output for an electronic device maycomprise displaying a set of graphical objects, each one of the set ofgraphical objects produced using a different level of vision correction,receiving a user selection of a graphical object from the set ofgraphical objects, and, in response to the user selection, identifying adisplay profile that is associated with the selected graphical object.The method may further comprise generating the graphical output inaccordance with the display profile, scanning at least a portion of aface of a user using a sensor, generating a depth map using the scan,and storing the depth map and associating the depth map with the displayprofile.

The method of providing a graphical output may further comprisedetermining, based on the user selection, that the user has a myopicvision condition and generating a new depth map based on a subsequentscan of the user. The method may further comprise determining, from thenew depth map, whether the user is wearing a corrective eyewear. Inaccordance with a determination that the user is wearing the correctiveeyewear, the method may cause a display to display the graphical output.

The method of providing a graphical output may further comprisedetermining, based on the user selection, that the user has a hyperopicvision condition and generating a new depth map based on a subsequentscan of the user. The method may further comprise determining, from thenew depth map, whether the user is wearing a corrective eyewear. Inaccordance with a determination that the user is not wearing thecorrective eyewear, the method may cause a display to display thegraphical output.

The method of providing a graphical output may further comprisedetecting an eye movement of the user using the sensor and, inaccordance with a determination that the eye movement corresponds to aneye strain condition, generating the graphical output.

The display profile may be one of a set of display profiles, eachdisplay profile may be associated with a different appearance of theuser, and each different appearance of the user may correspond to arespective corrective eyewear scenario.

Displaying the set of graphical objects may comprise presenting a set ofsuccessive screens to the user. Each one of the successive screens maycontain one or more graphical objects of the set of graphical objects.

The method of providing a graphical output may further comprisedetermining, from the user selection, a visual acuity of the user anddisplaying information regarding the visual acuity to the user.

An electronic device may comprise a housing, a display positioned atleast partially within the housing and configured to display a graphicaloutput, a transparent cover positioned at least partially over thedisplay, an optical sensor positioned below the transparent cover andconfigured to obtain a scan of at least a portion of a face of a user,and a processor.

The processor may be configured generate a depth map using the scan, anddetermine a similarity score between the depth map and one or moreidentity maps of a set of stored biometric identity maps that areassociated with a registered user.

The processor may be additionally configured to, in response to thesimilarity score exceeding a threshold, identify the user as theregistered user, determine a corrective eyewear scenario using the depthmap, select a display profile that is associated with the correctiveeyewear scenario, and generate a graphical output in accordance with theselected display profile.

The optical sensor may comprise a light emitting module configured toproject a dot pattern on the portion of the face of the user and theoptical sensor may obtain the scan of the portion of the face of theuser using the projected dot pattern.

The projected dot pattern may be produced by a series of infrared lightrays emitted from the light emitting module toward the portion of theface of the user and the optical sensor may further comprise aninfrared-sensing array configured to detect infrared light reflectedfrom the portion of the face of the user.

The corrective eyewear scenario may correspond to the registered userwearing a corrective eyewear. In some embodiments, the correctiveeyewear scenario may correspond to the registered user not wearing acorrective eyewear.

The corrective eyewear scenario may correspond to the registered userwearing a privacy eyewear and the graphical output may include a privacyblur that appears unblurred when viewed using the privacy eyewear.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to representative embodiments illustrated inthe accompanying figures. It should be understood that the followingdescriptions are not intended to limit the embodiments to one preferredembodiment. To the contrary, it is intended to cover alternatives,modifications, and equivalents as may be included within the spirit andscope of the described embodiments as defined by the appended claims.Identical reference numerals have been used, where possible, todesignate identical features that are common to the figures.

FIG. 1 illustrates a side view of an electronic device performing afacial recognition scan, as described herein.

FIG. 2A illustrates a front view of an electronic device displaying astandard graphical output, as described herein.

FIG. 2B illustrates a front view of an electronic device displaying avision-corrected graphical output, as described herein.

FIG. 3A illustrates a front view of an electronic device displaying avision setting menu, as described herein.

FIG. 3B illustrates a front view of an electronic device displaying avision diagnostic test, as described herein.

FIG. 4 depicts an example method of associating a particular graphicaloutput with an appearance of a user, as described herein.

FIG. 5 depicts an example process of controlling a vision-correctingoperation of an electronic device, as described herein.

FIG. 6 depicts an example process of a vision diagnostic operation and apresentation of a graphical output for a user with myopic vision andbased on the presence of corrective eyewear, as described herein.

FIG. 7 depicts an example process of a vision diagnostic operation and apresentation of a graphical output for a user with hyperopic vision andbased on the presence of corrective eyewear, as described herein.

FIG. 8 depicts an example process of an automatic vision diagnosticoperation and a control of a graphical output, as described herein.

FIG. 9 depicts an example process of generating and displaying a privacyscreen in response to a facial scan of a user, as described herein.

FIG. 10 depicts an example block diagram of an electronic device thatmay perform the disclosed processes and methods, as described herein.

The use of cross-hatching or shading in the accompanying figures isgenerally provided to clarify the boundaries between adjacent elementsand to facilitate legibility of the figures. Accordingly, neither thepresence nor the absence of cross-hatching or shading conveys orindicates any preference or requirement for particular materials,material properties, element proportions, element dimensions,commonalities of similarly illustrated elements, or any othercharacteristic, attribute, or property for any element illustrated inthe accompanying figures.

Additionally, it should be understood that the proportions anddimensions (either relative or absolute) of the various features andelements (and collections and groupings thereof), and the boundaries,separations, and positional relationships presented therebetween, areprovided in the accompanying figures merely to facilitate anunderstanding of the various embodiments described herein, may notnecessarily be presented or illustrated to scale, and are not intendedto indicate any preference or requirement for an illustrated embodimentto the exclusion of embodiments described with reference thereto.

DETAILED DESCRIPTION

The following disclosure relates to various techniques for generating,providing, and displaying various graphical outputs, including astandard graphical output and a vision-corrected graphical output, on adisplay of an electronic device. As described herein, a “standardgraphical output” may be used to refer to a graphical output of agraphical user interface that appears undistorted to a user havingnormal eyesight without the use of corrective eyewear (e.g., not near-or far-sighted). As described herein, a “vision-corrected graphicaloutput” may be used to refer to a graphical output of a graphical userinterface that has been adapted to improve the clarity of the output fora user having a vision deficiency or a user that is viewing the screenusing a corrective lens not adapted for close viewing. Thevision-corrected graphical output may be selectively or entirelyblurred; may overlay a, for example, grid-like filter over a standardgraphical output; may present the graphical output and/or certaingraphical elements as larger, brighter, with a different color palate,and the like; and may include other vision-correcting techniques.

As discussed herein, a user may have difficulty perceiving a standardgraphical output depending on the user's visual acuity and depending ona corrective eyewear scenario of the user. As used herein, a “correctiveeyewear scenario” may refer to a presence, or lack of presence, ofcorrective eyewear on a user's face. Depending on a corrective eyewearscenario of the user, the user may have difficulty perceiving a standardgraphical output. For example, a myopic (e.g., nearsighted) user may beable to easily perceive a standard graphical output while not wearingcorrective eyewear, but may have difficulty perceiving the standardgraphical output while wearing corrective eyewear (e.g., the correctiveeyewear may improve the user's vision for far away objects whilehindering the user's vision for nearby objects). Likewise, a hyperopic(e.g., farsighted) user may experience the opposite effect and may beable to easily perceive a standard graphical output while wearingcorrective eyewear, but may have difficulty perceiving the standardgraphical output while not wearing corrective eyewear.

In various embodiments presented herein, an electronic device maypresent a vision-corrected graphical output to a user who wouldordinarily have difficulty perceiving a standard graphical output.Further, an optical sensor system may detect the presence of correctiveeyewear and may switch between a standard graphical output and avision-corrected graphical output depending on whether a user is wearingthe corrective eyewear or not, as determined by comparing the user'sappearance with a set and/or subset of identity maps, as describedherein. For example, an electronic device may present a myopic user witha standard graphical output when the user is not wearing correctiveeyewear and may present the user with a vision-corrected graphicaloutput when the user wearing the corrective eyewear. In someembodiments, many display profiles may be associated with a single userhaving multiple corrective eyewear.

In various embodiments, a vision-corrected graphical output includesgraphical elements of a graphical user interface that appears clear to auser having a visual deficiency and/or may otherwise correspond tovarying levels of vision correction. Many different types ofvision-corrected graphical outputs may be provided and eachvision-corrected graphical output may correspond to a different visioncondition of a user and/or to a presence of corrective eyewear worn by auser, as determined by a facial scan of a user's face by, for example,an optical sensor system. For example, a user who has hyperopic vision(e.g., a user who is farsighted), may have difficulty perceiving nearbyobjects (e.g., a display on a mobile phone) without the use ofcorrective eyewear, but may be able to easily perceive the same nearbyobjects when wearing the corrective eyewear. For such a user, theelectronic device may present a standard graphical output when the useris wearing the corrective eyewear and may provide a vision-correctedgraphical output when the user is not wearing the corrective eyewear. Assuch, a vision-corrected graphical output corresponding to a myopicvision deficiency may exhibit different characteristics than avision-corrected graphical output corresponding to a hyperopic visiondeficiency. Many types of visual deficiencies are considered, includingmyopia, hyperopia, astigmatism, presbyopia and higher-order aberrationsthat may be difficult to correct with eyeglasses. The systems andtechniques described herein may also be used to account for differentvision-related perception issues that may not be formally classified asa vision deficiency. For example, the systems and techniques describedherein may be used to account for a user's aversion or preference tobright or flashing light sources, aversion or preference to a particularcolor, tint, or hue, or other vision-related aversion or preference. Thefollowing techniques may also be applied to help compensate forcolorblindness or other visual perception issues.

As discussed herein, many such vision-corrected graphical outputs,corresponding to different vision deficiencies, are considered.Specifically, a display may be driven in accordance with a displayprofile or a display setting profile that is used to produce, what isreferred to herein as a vision-corrected graphical output. In someembodiments, a vision-corrected graphical output may be produced on adisplay of an electronic device by pre-sharpening a two-dimensionalimage presented on the display using the inverse point spread functionof a user's eye. Some embodiments of a vision-corrected graphical outputinclude a multilayer display with prefiltered image generation, atailored, ray-traced display that produces a light field via lensletarrays, and a combination light field display with computationalprefiltering. Additionally or alternatively, a four-dimensional lightfield display with parallax barriers and/or lenslet arrays may be usedto counteract a user's vision deficiency. In some embodiments,refractions and aberrations of a user's eye, as derived from a user'sprescription, may be mathematically modeled and a processor of anelectronic device may use the mathematical representation to present afour-dimensional light field, via a display, to the user so that atwo-dimensional image with high clarity is perceived by the user. Insome embodiments, a pinhole parallax barrier may be provided, eitherdigitally or as a physically separate barrier, to optimize lightthroughput. These techniques are provided by way of example and othertechniques not expressly listed above, may be used to produce avision-corrected graphical output.

In some cases, the vision-corrected output is adapted to account fordifferent levels of vision correction in each of the user's eyes. Forexample, multiple outputs, each output configured to provide a differentlevel or different type of vision compensation, may be presented to theuser as a composite vision-corrected output. In some embodiments,multiple images may be presented at different angles to the user at thesame time. In this way, the user may perceive a three-dimensional imageon the two-dimension display. Additionally or alternatively, themultiple images presented at different angles may be individuallygenerated to compensate for vision deficiencies of individual eyes. Forexample, a user may have a certain prescriptive value corresponding to avision deficiency in their left eye and may have a differentprescriptive value corresponding to a different vision deficiency intheir right eye. By providing two different graphical outputs, eachcorresponding to a different vision deficiency, at different angles, thevision deficiencies of both of a user's eyes may be counteracted.Examples of such displays, which may be referred to as glasses-freethree-dimensional or light field displays, include volumetric displays,multifocal displays, and super-multi-view displays and can create theillusion of a three-dimensional object and/or can correct for the visiondeficiencies of different eyes.

In some embodiments, a vision-corrected graphical output may result inlarger text and/or images; a high contrast between graphical elements; acolor-shifted display to make certain graphical objects clearly visible;a simplified graphical output not including complex graphical images;and the like. In some embodiments, a vision-corrected graphical outputmay user alternative colors to correct for colorblindness. For example,a user with red-green colorblindness may be ordinarily unable toperceive the colors red and green. In a color-shifted example, elementsthat would ordinarily appear as red or green may appear as a differentcolors easily perceivable to the colorblind user.

In some embodiments, the aforementioned visual settings may be producedin response to environmental information detected by sensors on anelectronic device. For example, a high-sunlight environment may producean undesired glare on a display of the electronic device. Opticalsensors may detect the high brightness level and may perform variousdisplay adjusting operations to allow a user to more easily perceive thedisplay, as described herein.

As noted above, a particular vision-corrected graphical output maycorrespond to a vision condition of a user. With the use of correctiveeyewear (e.g., eyeglasses), a user's visual perception may changedepending prescriptive value of the corrective eyewear. For example, auser with a hyperopic vision condition may have prescription glasses tocompensate for the user's natural vision condition. In light of theuser's changing visual perception, a user's perception of a display ofan electronic device may also individually vary depending on whether theuser is wearing corrective eyewear or not and depending on the user'svision condition. In some users, wearing corrective eyewear may assistin viewing a display. In other users, wearing corrective eyewear mayhinder viewing a display.

In some embodiments, a camera or other types of optical sensor may beprovided on an electronic device to scan at least a portion of a user'sface corresponding to the user's facial biometrics. As used herein, theterm “optical sensor” includes any type of sensor configured to sense ordetect light, such as a camera (e.g., a forward-facing camera, a videocamera, an infrared camera, a visible light camera, a rear-facingcamera, a wide-lens camera, any combination thereof, and the like); anambient light sensor; a photodiode; a light detector; a light detectionand ranging (LIDAR) detector; and/or any type of sensor that convertslight into an electrical signal. An optical sensor may additionally beprovided with a light emitter configured to project beams of light andmay capture image data of the projected beams of light. As discussedherein, an optical sensor may include any emitter, detector, and/or anysignal processing circuitry used to perform an optical detection and/oranalysis.

An optical sensor system, which may include a camera and a lightprojector, may be used to identify and/or authenticate a previouslyregistered user identity for access into an electronic device. Forexample, the optical sensor system may initiate a scan of a user's faceand may store a biometric identity map of the user's face in an internalstorage of the electronic device (e.g., as a facial registrationprocess). Thereafter, whenever the electronic device, via the opticalsensor system, determines that a scanned face shares a thresholdsimilarity with the biometric identity map, the electronic device mayallow a user to access the electronic device (e.g., the electronicdevice transitions from a “locked” state to an “unlocked” state). Theoptical sensor may be used to identify or authenticate a user in orderto perform restricted functions including, for example, onlinepurchases, software downloads, application logins, restricted fileaccess, and/or other device operations.

In some embodiments, additional or alternate sensors may be used toperform an identity recognition process and/or an initiation of a scanof a user's face. For example, sensors using sound propagation tolocation and map an object, including, for example, detectors utilizingsonar, RADAR, ultrasonic detection, time-of-flight, and/or anycombination thereof, may be used in addition or instead of the opticalsensors as described herein. The type of sensor is not limited and anysensor capable of detecting facial features, either in three- ortwo-dimensions may be used.

In some embodiments, the electronic device may further direct an opticalsensor system to scan at least a portion of a user's face after theuser's identity is confirmed and may store facial data corresponding toalternate appearances of the user. The alternate appearance may bestored as an alternative biometric identity map as a set or subset ofidentity maps that are associated with the appearance of the user (e.g.,a corrective eyewear scenario in which the user is wearing or notwearing corrective eyewear). In addition to identifying and/orauthenticating the user, the optical sensor system may determine acorrective eyewear scenario of the user using one or more of thealternative biometric identity maps created based on a user's previousalternative appearance. For example, when a user is wearing correctiveeyewear, the electronic device may confirm the user's identity (using anormal identification procedure) and may store a depth map of the userwith the corrective eyewear as an alternate biometric identity map or asa subset of a stored biometric identity maps that are associated with aregistered user. Many different alternate appearances may be associatedwith one user. For example, one alternate appearance may be associatedwith a user wearing prescription glasses with a rectangular frame,another alternate appearance may be associated with user wearing readingglasses with a circular frame, another alternate appearance may beassociated with a user wearing sunglasses, another alternate appearancemay be associated with a user wearing no glasses, another alternateappearance may be associated with user with white-framed glasses, and soon.

Each alternate appearance of the user may be associated captured by anoptical sensor system, may be used to generate a subset of identitymaps, and may be associated with a particular corrective eyewearscenario. A particular display profile may further be associated withthe corrective eyewear scenario. As used herein, a “display profile” mayrefer to a profile that is used to generate a standard graphical outputor a vision-corrected graphical output. The display profile may beselected or identified based on the particular biometric identity mapthat is used to identify or authenticate the user. In this way, thefacial identification operation may be used to adapt the graphicaloutput of the display to be more easily readable by the user.

In some cases, a user wearing corrective eyewear may interact with avision diagnostic process on an electronic device to determine a visualacuity of the user, as discussed herein. Once the visual acuity of theuser is determined, a vision-corrected graphical output may bedetermined to be the graphical output that the user most easilyperceives. Further, the optical sensor system may scan at least aportion of the user's face and may detect an alternate appearance of theuser. A depth map, or a set of depth maps, may be created from thedetected alternate appearance and may be associated with thevision-corrected graphical output corresponding to the user's determinedvisual acuity. This association may be stored as a vision-correctingdisplay profile within, for example, a memory of an electronic device oron a distributed computing system. Thereafter, whenever the alternateappearance of the user is detected by the optical sensor system, theelectronic device may automatically present the vision-correctedgraphical output to the user.

As discussed herein, a standard graphical output may additionally beassociated with one or multiple depth maps created from one or multiplealternate appearances of the user within one or multiple displayprofiles. Further, a single user may be associated with many displayprofiles. A user may interact with a vision diagnostic process multipledifferent times, each time having a different appearance. In this way,multiple display profiles may be created. Further, as used herein, a“standard display profile” may refer to a display profile that outputs astandard graphical output and a “vision-correcting display profile” mayrefer to a display profile that outputs a vision-corrected graphicaloutput.

In some embodiments, depth maps created from a particular appearance ofa user may be associated with display profiles containing instructionsfor a vision-corrected graphical output without directly interactingwith the electronic device in a vision diagnostic process. For example,systems of the electronic device may monitor a user's interactionsduring normal interactions with the electronic device and may perform avision diagnostic analysis based on a user's interaction history. Manyother ways of diagnosing a user's vision perception are considered andare described herein.

As described herein, a “depth map” may refer to a two-dimensional imagewith depth information. For example, an optical sensor including aprojector may project a series of infrared light beams to a user's face.The infrared light beams may be captured as a series of infrared dotsand depth information may be determined by measuring a distance betweeneach infrared dot. Once the infrared dots are captured as an infraredimage, a mathematical representation may be created by transforming theinfrared image via a facial algorithm. In some embodiments, the numberof infrared light beams/infrared dots may be more than 10,000. In someembodiments, the number of infrared light beams/infrared dots may be30,000 or more than 30,000. In some embodiments, a depth map may referto a three-dimensional image with X-, Y-, and Z-planes.

As described herein, “corrective eyewear” may refer to any type ofeyeglasses, lenses, or eyewear such as, for example, contact lenses,reading glasses, sunglasses, prescription eyewear, non-prescriptioneyewear, and the like. Though the term “corrective” is used, correctiveeyewear, as described herein, include glasses or lenses without anyparticular corrective properties, including ordinary glass lenses.Further, the corrective eyewear are not limited to lenses themselves,but may include frames, chains, and other structural elements associatedwith corrective eyewear.

As described herein, the optical sensor system may be any appropriateoptical sensing system and may include any computer element or system(e.g., a projector, a light emitter, a camera, an optical sensor, amachine learning module, computer code, and the like). In someembodiments, an optical sensing system may be a camera configured tocapture two-dimensional images of a face of a user. The two-dimensionalimages may correspond to how a user is typically perceived in a visiblelight spectrum. Machine learning techniques and/or other methods oftwo-dimensional image analysis may then be used to perform a facialrecognition process, as discussed herein. In some embodiments, theoptical sensing system may be a facial recognition system and mayinclude a light emitting module configured to project a dot pattern ontoa user's face and a camera configured to capture image information ofthe projected dot pattern.

These and other embodiments are discussed with reference to FIGS. 1-10.However, those skilled in the art will readily appreciate that thedetailed description given herein with respect to these figures is forexplanatory purposes only and should not be construed as limiting.

FIG. 1 illustrates a side view of an example electronic device 100performing a facial scan of a user 106 and using the optical sensorsystem 102, in accordance with some described embodiments. Theelectronic device 100 may additionally include a graphical userinterface presented on a display 110 of the electronic device 100.

As depicted in FIG. 1, the optical sensor system 102 may include a lightemitting module 103 (e.g., a projector) and a front-facing camera 105.In some embodiments, the optical sensor system 102 may be used as afacial recognition system. The light emitting module 103 may beconfigured to emit light rays 104 in accordance with a dot pattern. Thelight rays 104 may be light in any electromagnetic spectrum, such as,for example, light in the visible spectrum, light in the infraredspectrum, or light in the ultraviolet spectrum. When the light rays 104are projected toward a three-dimensional object, such as a face of theuser 106, the resulting dot pattern may conform to the contours of thethree-dimensional object, due to some of the light rays 104 reaching theface of the user 106 before other light rays 104. As a result, the dotsin the dot pattern are not spaced equally. The front-facing camera 105(e.g., a camera designed to detect visible, infrared, and/or ultravioletlight, not shown) may capture images of the dot pattern (e.g., portionsof the dot pattern that are reflected from the user's face toward thefront-facing camera 105) and may create a biometric identity depth mapand/or a set of biometric identity maps of the face of the user 106based on the spacing between the individual dots. The light rays 104depicted in FIG. 1 may represent a fraction of the total light rays andthe light emitting module 103 may emit more light rays 104 than thosedepicted in FIG. 1. Though “biometric identity depth map” is singular,many different biometric identity depth maps may be created and anaggregation of the biometric identity depth maps may be used to confirma user's identity. Similarly, any described “depth map” may be a singledepth map or may be comprised of multiple depth maps.

In some embodiments, a biometric identity depth map may be createdand/or stored within a memory of the electronic device 100. The storedbiometric identity depth map may correspond to the user 106 and the user106 may be considered registered by the optical sensor system 102 and/orthe electronic device 100. Thereafter, whenever the registered user(e.g., the user 106) is detected by the optical sensor system 102, anadditional depth map is, or additional sets of depth maps are, generatedusing the optical sensor system 102. The additional depth map is thencompared with the stored biometric identity depth map. If the additionaldepth map shares a similarity score with the stored biometric identitymap, the user is identified and/or authenticated and permitted access tothe electronic device 100. In some embodiments, one or more identitymaps from the stored biometric identity maps are used to authenticate oridentify a registered user.

In some embodiments, any of the generated depth maps (e.g., the storedbiometric identity maps or one or more of the identity maps from thestored biometric identity maps) may be configured to change, vary,and/or evolve over time. For example, if a user initially registers witha clean-shaven appearance, the initial stored biometric identity mapsmay correspond to the user with the clean-shaven appearance. If the userthen begins to grow facial hair, the stored biometric identity maps mayvary in accordance with the growing facial hair. In some embodiments,the stored biometric identity maps may vary in response to detecting auser's face at an unlocking procedure, generating depth maps based onthe user's face, determining that the depth maps share a high similaritywith the initial stored biometric identity maps, and updating theinitial stored biometric identity maps to account for the user's changein appearance. By making incremental changes to the stored biometricidentity maps, a user's identity may be confirmed even after changes inthe user's appearance. For significant changes to the user's appearance,systems of the electronic device may require an affirmative userinteraction (e.g., entering a password) before the stored biometricidentity maps are updated.

As described herein, the front-facing camera 105 may be either onecamera (as depicted in FIG. 1) or may be a number of cameras. Forexample, the front-facing camera 105 may include an infrared detectingfront-facing camera and a visible light detecting front-facing camera.The optical sensor system 102, light emitting module 103, andfront-facing camera 105 may be positioned behind the display 110 of theelectronic device 100, as indicated in FIG. 1 by the presence of dashedlines.

Certain features of the face of the user 106 are typical for humanfaces, including eyes, ears, lips, a nose, and eyes, but may vary inshape and/or size depending on each individual. Based on the createdbiometric identity depth map and on these common features, an identityprofile of a user may be created and stored on the electronic device 100for identity recognition purposes. Thereafter, the optical sensor system102 may perform identification and/or authentication processes thatunlocks an operation of the electronic device 100 when a subsequentlyscanned face shares a threshold similarity score with the createdbiometric identity depth map.

Further, corrective eyewear 108 may be worn by a user 106. Thecorrective eyewear 108 be included in an identity depth map of the faceof the user 106. For example, the light rays 104 may be projected towardthe face of the user 106 but may be intercepted by the correctiveeyewear 108 before reaching the face. Portions of the corrective eyewear108 that are transparent to the light rays 104 may permit the light rays104 to reach the face of the user 106. The front-facing camera mayreceive portions of the light rays 104 that reflect from the face of theuser 106 and/or from the corrective eyewear 104 and may use the receivedportions of the light rays 104 to generate one or more depth maps, asdescribed herein. Though a portion of the light rays 104 may beintercepted, a user's identity may still be determined if the visiblefacial features of the user meet the threshold similarity score with thecreated biometric identity depth map.

In some embodiments, the corrective eyewear 108 may be incorporated intoan alternate appearance depth map. For example, a user with a confirmedidentity may nevertheless exhibit different facial appearances (e.g., auser without the corrective eyewear 108 would look differently than theuser with the corrective eyewear 108). Each different facial appearancemay all be considered to correspond to the same user 106 (e.g., eachfacial appearance may meet the threshold similarity score with thecreated biometric identity depth map), but may nevertheless result in adifferent, alternate appearance depth map. The alternate appearancedepth maps may be further categorized into subsets of the biometricidentity depth map and may each correspond to a different alternateappearance and/or corrective eyewear scenario.

To distinguish between an identity confirmation and an alternateappearance, two different similarity thresholds may be used. The firstsimilarity threshold may be related to a first similarity score that ameasured depth map shares with a pre-registered biometric identity depthmap. The second similarity threshold may be related to a secondsimilarity score that a measured depth map shares with a pre-registeredalternate appearance depth map. In some embodiments, the firstsimilarity score may be met (in order to confirm a user's identity)before the second similarity score is measured. As the user's identitymay already be confirmed before the second similarity score is measured,a minimum second similarity score may be lower (e.g., less stringent)than a minimum first similarity score.

In some embodiments, once a user's face is scanned, the scanned face maybe categorized into a number of alternate appearance depth maps. Anexample of this system follows. During a facial enrollment process, theoptical sensor system 102 may scan a user's face a number of times andmay store the scans of the user's face in an identity profile. In someembodiments, the scans may be a set of biometric identity maps and theset of biometric identity maps may be stored in the identity profile.Thereafter, whenever the user's face is scanned by the optical sensorsystem 102, the optical sensor system 102 may compare the scanned facewith the set of biometric identity maps in the identity profile and mayconfirm the identity of the user when a scanned face of the user sharesa similarity score with the set of biometric identity maps in theidentity profile.

A subset of scans within the identity profile may additionally becreated through the use of the electronic device 100. For example, auser may wear three different types of corrective eyewear. One type maycorrespond to reading glasses, one type may correspond to prescriptionglasses, and one type may correspond to sunglasses. In this way, theuser may have four different appearances (three appearances for eachtype of glasses and one appearance with no glasses) each associated withthe same identity.

Scans and/or depth maps, as described herein, corresponding to eachdifferent appearance may be separated as subcategories within theidentity profile. For example, scans of a user wearing sunglasses mayshare a suitable similarity score with a preregistered scanned face, soas to confirm an identity, but may share enough differences so as to bestored as an alternate appearance. Thereafter, whenever the user wearsthe sunglasses and is scanned by the optical sensor system 102, theoptical sensor system 102 may determine both the user's identity (bycomparing the scanned face with the scans in the larger identityprofile) and may determine the user's alternate appearance (by comparingthe scanned face with the scans in the subset of alternate appearancescans). In this way, a number of alternate appearance depth maps eachcorresponding to the same user may be created and stored as alternateappearance profiles within the identity profile.

FIG. 2A is an illustration of a standard graphical output presented on adisplay 210 of an electronic device 200 and the electronic device 200,as described herein. The electronic device 200 includes an opticalsensor system 202, which may include substantially similar modules(e.g., a light emitting module 203 and a camera 205) as discussed abovewith reference to FIG. 1. As discussed herein, the optical sensor system202 may, in some embodiments, be used as a facial recognition system.The electronic device 200 may include housing 214 within which thedisplay 210 is provided. Presented on the display 210 is a graphicaluser interface presenting a graphical output. The graphical userinterface may also include a calibration graphic 212.

As discussed herein, the calibration graphic 212 may be a graphicalelement with which a user may interact. If the user perceives thegraphical output as blurry or otherwise not clear (due to a visiondeficiency of the user), the user may interact with the calibrationgraphic 212 to vary a visual condition of the graphical output. Forexample, the calibration graphic 212 may direct the graphical output tovary in a way that compensates for a common vision deficiency (e.g., amyopic vision deficiency). If the user presses the calibration graphic212 a second time, the calibration graphic 212 may direct the graphicaloutput to produce a vision-corrected graphical output in a way thatcompensates for another common vision deficiency (e.g., a hyperopicvision deficiency). In this way, the user may press the calibrationgraphic 212 consecutively to cycle through a number of vision-correctedgraphical outputs. FIG. 2B depicts an example of such a vision-correctedgraphical output.

In some embodiments, a user may interact with the calibration graphic212 to intentionally blur the graphical output presented on the display210. For example, if a user desires privacy or does not want a nearbyperson to view what is presented on the display 210, the user mayinteract with the calibration graphic 212 to make the graphical outputillegible. In some embodiments, e.g., as depicted and described withrespect to FIG. 9, privacy eyewear may counteract the intentional blur.

The manner in which a user interacts with the calibration graphic 212 isnot limited. The user may utilize a touch sensitive display tophysically press the display 210 in a region above the calibrationgraphic 212. Alternatively, the user could utilize any number of inputdevices (e.g., a mouse or keyboard) to select the calibration graphic212.

As presented herein, FIG. 2A may correspond to a standard graphicaloutput on the display 210. The standard graphical output may bepresented to the user based on the results from the optical sensorsystem 202. For example, the optical sensor system 202 may scan at leasta portion of the user's face to generate a scan, generate a depth mapfrom the scan, determine a similarity score between the depth map andone or more biometric identity maps of a set of stored biometricidentity maps that are associated with a registered user. In response tothe similarity score exceeding a threshold, the user may be identifiedand/or authenticated as the registered user and a corrective eyewearscenario may be determined using the depth map. Further, avision-correcting display profile associated with the corrective eyewearscenario may be selected and vision-corrected graphical output may begenerated in accordance with the selected vision-correcting displayprofile. For example, a user wearing corrective eyewear may interactwith the standard graphical output as depicted in FIG. 2A. As the useris interacting with the standard graphical output, the optical sensorsystem 202 may scan the user's face, as described herein, may create aset of depth maps and may link the set of depth maps with the standardgraphical output. Thereafter, whenever the user is wearing correctiveeyewear and a scan sharing a similarity score with the set of depth mapsis captured by the optical sensor system 202, the standard graphicaloutput may be automatically displayed on the display 210.

As discussed with respect to FIG. 1, each alternate appearance of theuser may be stored as a set of scans and/or depth maps in subcategoriesof an identity profile. Each one, or multiple, of these subcategoriesmay be linked to a standard display profile or a vision-correctingdisplay profile.

For example, a user may have five different appearances, one withoutglasses, one with prescription glasses, one with non-prescriptionsunglasses, one with prescription sunglasses, and one with readingglasses. For two of those appearances (e.g., when the user is wearingthe prescription glasses and the prescription sunglasses), the user maymost clearly perceive a standard graphical output. As such, thedisclosed system may capture a set of scans, may generate depth mapscorresponding to the two appearances, and may associate the set of scansand/or depth maps of both the user wearing the prescription glasses andthe user wearing the prescription sunglasses with the standard graphicaloutput in a standard display profile. Thereafter, whenever the user isdetected as wearing either of the two aforementioned glasses, thestandard graphical output may be presented. For the other threeappearances, the user may most clearly perceive a vision-correctedgraphical output. As such, depth maps corresponding to the other threeappearances may be associated with a vision-correcting display profileand a vision-corrected graphical output may be displayed to the userwhen any of the three appearances are detected by the optical sensorsystem 202.

In some embodiments, the user may change which display profilecorresponds to a particular set of depth maps corresponding to aparticular appearance. For example, a user wearing the prescriptionsunglasses in the above example may begin to experience difficulty inperceiving the standard graphical output. The user may, for example,press calibration graphic 212 while wearing the prescription sunglassesto change the standard graphical output into a vision-correctedgraphical output. Once the vision-corrected graphical output isdisplayed (see FIG. 2B, below), the optical sensor system 202 may scanthe user's face and may store scans and/or depth maps corresponding tothe user wearing the prescription sunglasses with the presentedvision-corrected graphical output and a vision-correcting displayprofile. Thereafter, display profiles containing instructions for thevision-corrected graphical output may be associated with the storedscans and/or depth maps corresponding to the user wearing theprescription sunglasses in the vision-correcting display profile.

In some embodiments, the optical sensor system 202 may scan the user'sface after the user has spent a threshold amount of time interactingwith a standard or vision-corrected graphical output. In someembodiments, the optical sensor system 202 may scan the user's faceimmediately after a standard or vision-corrected graphical output isfirst presented. In some embodiments, if a user has spent a certainamount of time interacting with the electronic device while in a certaindisplay mode, a processor of the electronic device may determine thatthe displayed graphical output is well perceived by the user and mayassociate a display profile containing instructions to generate thegraphical output with scans and/or depth maps corresponding to theuser's appearance as a result of the determination.

FIG. 2B is an illustration of a vision-corrected graphical outputpresented on the display 210 of the electronic device 200 and theelectronic device 200, as described herein. In some embodiments, thevision-corrected graphical output is displayed to the user after theuser presses the calibration graphic 212 as depicted in FIG. 2A.

In FIG. 2B, the standard graphical output is varied in order tocompensate for a vision deficiency. As shown in FIG. 2B, the graphicaloutput is presented in a blurred fashion. Additionally, the calibrationgraphic 212 is blurred into a blurred calibration graphic 212 a. Theuser may further press the blurred calibration graphic 212 a in order tocontinue cycling through a number of pre-set vision-corrected graphicaloutputs and/or to return to the standard graphical output.

In some embodiments, a blurred graphical output may be provided thatmakes the graphical output of FIG. 2B illegible. For example, if a userdesires privacy or does not want a nearby person to view what ispresented on the display 210, the blurred graphical output may beproduced to make the graphical output illegible. In some embodiments,e.g., as depicted and described with respect to FIG. 9, privacy eyewearmay counteract the otherwise illegible graphical output.

Alternatively or additionally, the vision-corrected graphical output maybe presented to a user when the optical sensor system 202 detects analternate appearance and depth maps generated from the alternateappearance corresponds to previously stored depth maps associated withdisplay profiles containing instructions for the vision-correctedgraphical output in a display profile (e.g., a vision-correcting displayprofile).

For example, as discussed with respect to FIG. 2A, a user may have fivedifferent appearances, one without glasses, one with prescriptionglasses, one with non-prescription sunglasses, one with prescriptionsunglasses, and one with reading glasses. If the user has hyperopicvision (e.g., the user is farsighted), the user may most easily perceivethe standard graphical output of FIG. 2A while wearing the prescriptionsunglasses and the prescription glasses. Likewise, the user may mosteasily perceive the vision-corrected graphical output of FIG. 2B whilenot wearing glasses, wearing non-prescription sunglasses, and wearingreading glasses. As such, the two prescription glasses may be associatedwith the standard graphical output as part of a standard display profileand the three other appearances may be associated with display profilescontaining instructions for the vision-corrected graphical output.Thereafter, whenever the optical sensor system 202 detects the twoprescription glasses, scans and/or depth maps generated from the twoprescription glasses may correspond to a standard display profile andthe standard graphical output of FIG. 2A may be displayed. When theoptical sensor system 202 detects the other three appearances, scansand/or depth maps generated from any of the three appearances maycorrespond to a vision-corrected display profile and thevision-corrected graphical output of FIG. 2B may be displayed. Thedisclosed system is not limited to the above example and any number ofscans and/or depth maps of a variety of user appearances may beassociated with any number of display profiles. Further, multiplevision-correcting display profiles may be provided, eachvision-correcting-display profile related to a different visualperception of the user while wearing different corrective eyewear.

FIG. 3A illustrates an example settings screen for a vision setting menuon a display 310 of an electronic device 300. The electronic device 300includes an optical sensor system 302, which may include substantiallysimilar modules (e.g., a light emitting module 303 and a camera 305) asdiscussed above with reference to FIGS. 1 and 2. As discussed herein,the optical sensor system 302 may, in some embodiments, be used as afacial recognition system. An electronic device 300 may include textboxes and sliders where a user may input an eyesight prescription or avision condition, as described herein. For example, the presented visionsetting menu may contain a text box labeled “Enter Prescription” andfour sliders labeled “Nearsighted,” “Farsighted,” “Presbyopia,” and“Astigmatism.” Each of the four, or more, sliders may correspond to adifferent vision condition.

If a user knows their personal visual prescription, the user may enter anumber (e.g., 20/200) into the provided text box labeled “EnterPrescription.” A long-form prescription may also be provided (e.g., apop-up box may appear with spaces for a user to fill in withprescription information). The long form prescription may includeinformation for an O.D. eye (e.g., oculus dexter, or right eye) and anO.S. eye (e.g., oculus sinister, or left eye). The long-formprescription may additionally include, for example, sphere, cylinder,axis, add, prism, and additional information.

Once the user enters the prescription information, the provided slidersmay automatically shift based on the prescription information. Forexample, a prescription indicative of nearsighted-ness may automaticallyshift the nearsighted slider to the “ON” position (e.g., to the right inFIG. 3A). The provided sliders may also be individually controllable bya user. If a user does not know their exact prescription, but knowstheir underlying vision condition, the user may interact with thesliders without entering prescription information.

Once prescription information is entered into the vision setting menu, avision-corrected graphical output may be generated based on the userentered information. For example, if a user enters a prescription, aparticular vision-corrected graphical output may be generated based onthe prescription information. Further, based on the position of thesliders (e.g., the “nearsighted” and “farsighted” sliders), the systemmay provide a particular graphical output based on whether a user iswearing corrective eyewear or not, as discussed with respect to FIGS. 6and 7.

To compensate for particular vision deficiencies, a graphical output maybe distorted based on a number of visual distortion basis functions(e.g., Zernike polynomials). A myopic user may have spherical, coma,trefoil, and many other types of visual distortions. Similarly, ahyperopic, astigmatic, or otherwise visually deficient user mayexperience different levels of the same types of visual distortions.Each particular graphical output may be distorted based on these basisfunctions and these basis functions may be incorporated into any systemfor producing a graphical output, as described herein.

In some embodiments, if a “nearsighted” slider is in the “ON” positionand a “farsighted” slider is in the “OFF” position, the system maypresent the vision-corrected graphical output when scans and/or depthmaps corresponding to a user's wearing of corrective eyewear arereceived. Further, the system may present the standard graphical outputwhen scans and/or depth maps corresponding to the user's absence of worncorrective eyewear are received. In another example, if the“nearsighted” slider is in the “OFF” position and the “farsighted”slider is in the “ON” positon, the system may present the standardgraphical output when scans and/or depth maps corresponding to theuser's wearing of corrective eyewear are received and may present thevision-corrected graphical output when scans and/or depth mapscorresponding to the user's absence of worn corrective eyewear arereceived. The two examples provided are merely explanatory and anymanner of providing a graphical output with regard to a particularappearance of a user may be used.

FIG. 3B illustrates an example vision diagnostic test on a display 310of an electronic device 300. In the vision diagnostic test of FIG. 3B, anumber of graphical objects may be presented to the user along withassociated text (e.g., “Select the image that looks the clearest”). Eachgraphical object may have a different level of vision correction,similar to an eyesight test at an eye doctor's office. For example, afirst graphical object may be blurred to compensate for a myopic visiondeficiency, a second graphical object may be blurred to compensate for ahyperopic vision deficiency, a third graphical object may be enlarged tocompensate for an ocular degeneration, and a fourth graphical object maybe unaffected by any vision correction process.

In response to a user selection of a particular graphical object, avisual acuity of the user may be measured. Though only one screen isdepicted in FIG. 3B, many successive screens may be displayed to theuser in order to achieve a higher diagnostic confidence for the visiondiagnostic test. Each successive screen may contain different graphicalobjects with different levels of vision correction, as described herein.Once the visual acuity of the user is measured, information regardingvisual acuity of the user may be presented to the user (e.g., aprescription value may be presented to the user on the display). In thisway, the user may learn their visual acuity through the visiondiagnostic test.

The illustrations in FIG. 3B are merely explanatory and any manner ofproviding a vision diagnostic test may be used. For example, acollection of differently sized letters (e.g., a Snellen eye chart) maybe provided on the screen and a user may mark which letters are readableand which letters are not readable.

FIG. 4 depicts an example process 400 of associating a particulargraphical output with an appearance of a user, as determined by a facialscan.

At operation 402, the processor receives a display adjustment request.As discussed herein, the display adjustment request may be a usergenerated input (e.g., a user selects the calibration graphic 212 ofFIG. 2), an interaction with a vision setting menu (e.g., the visionsetting menu of FIG. 3A), and/or the like. In some embodiments, thedisplay adjustment request may be automatically determined by theelectronic device by, for example, tracking an eye movement of a userand determining when the user is experiencing visual strain.

At operation 404, the processor generates a vision-corrected graphicaloutput and displays the vision-corrected graphical output on a displayof the electronic device in response to the display adjustment request.The vision-corrected graphical output may correspond to a particularvision deficiency of the user (e.g., if the user has previously input aprescription value or a vision condition) or may be generated based onpredetermined defaults to compensate for a common vision deficiency. Insome embodiments, the generated vision-corrected graphical output mayreplace the previously displayed standard graphical output and may bepresented to the user instead of the standard graphical output.Additionally, the vision-corrected graphical output may only replacecertain graphical elements presented in the standard graphical output(e.g., a toolbar icon and/or graphical elements in a pulldown menu in agraphical user interface of the electronic device). In some embodiments,operations 402-404 may be repeated until the displayed vision-correctedgraphical output is acceptable to the user.

At operation 406, the processor may direct an optical sensor system(e.g., the optical sensor system 102) to perform a facial scan of theuser. The facial scan may be performed in a manner as discussed above(e.g., by creating a three-dimensional depth map from a projected dotpattern or by performing image analysis on a two-dimensional picture).Once a scan of the user's face is performed, the processor may generatea depth map from the scanned face and may determine that the generateddepth map shares a first similarity score with a pre-registered identitydepth map to confirm an identity of the user. If the identity isconfirmed, the processor may further determine if the depth mapcorresponds to a pre-registered alternate appearance, as discussedherein, by sharing a second similarity score. If no pre-registeredalternate appearance shares the second similarity score, then theprocessor may register the scan (or set of scans and/or depth maps) as anew alternate appearance depth map or set of depth maps within anidentity profile.

At operation 408, the processor may associate the scans and/or depthmaps generated from a face scanned at operation 406 with thevision-corrected output of operation 404. As discussed herein, a displayprofile may be created and may contain both data referring to the scansand/or depth maps of the scanned face and the vision-corrected output.Thereafter, whenever a depth map or set of depth maps generated from afacial scan of the user shares at least a minimum similarity score withthe face scanned at operation 406, the vision-corrected graphical outputmay be automatically displayed without any further user input.

In some cases, therefore, a user wearing a particular set of glasses maybe presented with a vision-corrected display designed to compensate fora vision deficiency of the user while wearing the glasses whenever theuser is detected wearing the glasses by an optical sensor system. Inother cases, a user not wearing any glasses may be presented with avision-corrected display designed to compensate for a vision deficiencyof the user while not wearing the glasses whenever the user is detectedas not wearing glasses by an optical sensor system.

The process 400 is an example process for associating a particulargraphical output with an appearance of a user and is not a limitingexample. Processes for associating graphical outputs with a user'sappearance may omit and/or add steps to the process 400. Similarly,steps of the process 400 may be performed in different orders than theexample order discussed above. In some embodiments, a vision-correctedgraphical output may be initially displayed to a user and a displayadjustment request may direct a processor to present a standardgraphical output as a modification and/or a replacement of thevision-corrected graphical output (e.g., in operations 402 and 404). Insome embodiments, a standard graphical output may be initially displayedto a user and a display adjustment request may direct a processor topresent a vision-corrected graphical output instead of the standardgraphical output as a modification and/or a replacement of the standardgraphical output (e.g., in operations 402 and 404).

FIG. 5 depicts an example process 500 of a vision-correcting operationof an electronic device. At operation 502, a processor performing theprocess 500 may direct an optical sensor to perform a facial scan of atleast a portion of a face of a user. As provided herein, an opticalsensor may project a number of light rays toward the user's face and maymeasure the distances of dots projected on the user's face. In someembodiments, the entirety of the user's face may be scanned, if able tobe detected by the optical sensor. In some embodiments, only a portionof the user's face (e.g., a nose, mouth and chin portion) may be scannedwith the optical sensor. In some embodiments, the scan may include anyobjects, such as corrective eyewear, worn on the user's face.

At operation 504, a depth map is generated using the scan conducted bythe optical sensor. The depth map may correspond to an appearance of theuser and may include three-dimensional information corresponding topeaks and valleys present on the user's face. One or any number of depthmaps may be generated. If a number of depth maps are generated, eachdepth map may correspond to a different angle of the user's face withrespect to the optical sensor. In some embodiments, the generated depthmap or depth maps may be stored within an internal or distributedmemory.

At operation 506, a processor may determine a similarity score betweenthe depth map or depth maps generated at operation 504 and one or morebiometric identity maps of a set of stored biometric identity maps thatare associated with a registered user. The set of stored biometricidentity maps may be depth maps that are created when a user isinitially scanned by the optical sensor during a registration processand may correspond to the user's identity.

In some embodiments, a similarity score may be determined by anysuitable statistical analysis and may include analysis related tostructural similarity measure (SSIM) approaches, feature similarityindex measure (FSIM) approaches, edge similarity measure approaches, andthe like. The similarity score may be measured by determining astatistical likelihood that the depth maps generated at operation 504reference the same user as the stored biometric identity maps. Once asimilarity score is determined, a processor may determine whether athreshold similarity value is met or surpassed. If the thresholdsimilarity value is met or surpassed, the user's identity may beconfirmed as the system has determined that the same user is referencesin both the depth maps created at operation 504 and the stored biometricidentity maps.

At operation 508, once the processor has determined that the similarityscore exceeds the threshold similarity score, the user may be identifiedand/or authenticated as the registered user. In some embodiments, oncethe user is identified and/or authenticated, the user may be permittedaccess into an internal memory of an electronic device and/or anelectronic device may transition from a locked state to an unlockedstate.

At operation 510, a corrective eyewear scenario may be determined usingthe depth map or depth maps generated at operation 504. The correctiveeyewear scenario may refer to the presence or absence of correctiveeyewear on the user's face. One such corrective eyewear scenario mayrelate to the user while the user is wearing corrective eyewear. Anothercorrective eyewear scenario may relate to the user while the user is notwearing corrective eyewear. In some embodiments, there may be multiplecorrective eyewear scenarios—each corresponding to a different style ofcorrective eyewear and/or to the absence of any corrective eyewear.

As discussed herein, any suitable statistical analysis, including SSIMand FSIM approaches, may be used to determine the presence or absence ofcorrective eyewear. In alternate or additional scenarios, the presenceor absence of corrective eyewear may be determined by differencesbetween the depth map generated at operation 504 and the storedbiometric identity maps. In some embodiments, a second similarity scoremay be used to determine the corrective eyewear scenario. The secondsimilarity score may be looser than the identification similarity scorediscussed above, as the second similarity score may be measured afterthe user's identification is confirmed.

At operation 512, the processor determines a vision-correcting displayprofile that is associated with the corrective eyewear scenariodetermined at operation 510. The vision-correcting display profile mayhave been previously registered with a corrective eyewear scenario. Forexample, a user may have previously identified that, when no correctiveeyewear is detected, a particular vision-correcting display profileshould be selected. In some embodiments, a vision-correcting displayprofile may refer to, by default, a particular corrective eyewearscenario.

At operation 514, a vision-corrected graphical output is generated inaccordance with the selected vision-correcting display profile. Thevision-corrected graphical output may correspond to a particular visiondeficiency of the user (e.g., if the user has previously input aprescription value or a vision condition) or may be generated based onpredetermined defaults to compensate for a common vision deficiency.

The process 500 is an example process for controlling avision-correcting operation for an electronic device and is not alimiting example. Processes for controlling a vision-correctingoperation may omit and/or add steps to the process 500. Similarly, stepsof the process 500 may be performed in different orders than the exampleorder discussed above.

FIG. 6 depicts an example process 600 of a vision diagnostic operationand a presentation of a graphical output for a user with myopic vision.As discussed above, a user with myopic vision (e.g., a nearsighted user)may clearly perceive nearby objects and may not clearly perceive farawayobjects. With the use of proper corrective eyewear (e.g., eyewear havinga suitable prescription), the user may be able to clearly perceive bothfaraway and nearby objects but may experience occasional eye strain whenviewing some nearby objects. The process 600 allows the user to perceivea graphical output in a manner best suited to the user's visual acuity.

At operation 602, the processor may perform a vision diagnostic processto identify a visual acuity of a user of the electronic device. Theuser's visual acuity may be determined by, for example, a visiondiagnostic test as described with respect to FIG. 3B. In someembodiments, a processor may receive a display adjustment request from auser generated input and may determine the user's visual acuity and/orvision condition based on the user's interaction with the electronicdevice. In some embodiments, the processor may determine the user'svisual acuity by a user input (e.g., entering the user's prescription ina setting, as depicted in FIG. 3A).

At operation 604, the processor may determine the user has myopiceyesight. The processor may determine that the user has myopic eyesightfrom the vision diagnostic process described with respect to operation602. In some embodiments, the processor may determine that the user bestperceives vision-corrected displays that have a myopic visioncorrection. In some embodiments, the processor may detect that a“Nearsighted” slider is in the “ON” position.

At operation 606, the processor may direct an optical sensor to performa facial scan of a face of the user. Here, the processor directs anoptical sensor system (e.g., the optical sensor system 102) to perform afacial scan of the user. The facial scan may be performed in a manner asdiscussed with respect to FIG. 1 (e.g., by creating a three-dimensionaldepth map from a projected dot pattern). Once a scan of the user's faceis performed, the processor may determine that the scanned face shares afirst similarity score with a pre-registered identity depth map toconfirm an identity of the user. If the identity is confirmed, theprocessor may further determine if the scanned face corresponds to apre-registered alternate appearance, as discussed herein, by sharing asecond similarity score.

At operation 608, the processor may determine a corrective eyewearscenario of the user. For example, the processor may determine whetherthe user is, or is not, wearing corrective eyewear. The processor maymake this determination by comparing the facial scan captured atoperation 606 with a pre-registered identity profile including a set ofalternate appearances. For example, one alternate appearance maycorrespond to the user wearing glasses and another may correspond to auser with no corrective eyewear. The processor may determine whichappearance corresponds most closely with the scan taken at operation 606(by, for example, determining two similarity scores between the scan andthe two appearances and determining which similarity score is thehighest).

At operation 610, the processor may determine that the user is notwearing corrective eyewear and may display a standard graphical output.Such a determination may be made in accordance with the scan taken atoperation 606 corresponding more closely to an appearance of the userwhere the user is not wearing eyewear.

At operation 612, the processor may determine that the user is wearingcorrective eyewear, may generate a vision-corrected graphical output,and may display the vision-corrected graphical output. Thevision-corrected graphical output may compensate for the myopic visionof the user by, for example, blurring a portion and/or the entirety of astandard graphical output; generating an overlay over a standardgraphical output; and/or making elements of a standard graphical outputlarger, brighter, and/or more distinct. In some embodiments, thegenerated vision-corrected graphical output may replace a previouslydisplayed standard graphical output and may be presented to the userinstead of a standard graphical output. Additionally, thevision-corrected graphical output may only replace certain graphicalelements presented in a standard graphical output. The vision-correctedgraphical output may be a default graphical output designed tocompensate for a myopic vision or may be generated based on theindividual prescription of the user.

The vision-corrected graphical output generated at operation 612 may beconfigured specifically to compensate for a myopic vision deficiency andmay differ from other vision-corrected graphical outputs, as describedherein. As a non-limiting example, a vision-corrected graphical outputgenerated to compensate for a myopic vision deficiency while a user iswearing corrective eyewear may be minimally different than a standardgraphical output. Such vision-corrected graphical output may, forexample, include elements that are larger, smaller, blurred, compressed,brighter, dimmer, color-shifted and/or stretched so that a myopic userwearing corrective eyewear may perceive the vision-corrected graphicaloutput as the user would when not wearing the corrective eyewear (e.g.,as the user perceives the standard graphical output as shown inoperation 610). The example above is merely explanatory and other typesof vision-corrected graphical outputs to compensate for a myopic visiondeficiency may be used.

In some embodiments, a vision-corrected graphical output may beinitially displayed to a user and the vision-corrected graphical outputmay continue being displayed at operation 612 and a standard graphicaloutput may be generated/modified and displayed at operation 610. In someembodiments, a standard graphical output may be initially displayed to auser and.the standard graphical output may continue being displayed atoperation 610 and a vision-corrected graphical output may begenerated/modified and displayed at operation 612.

The process 600 is an example process for a vision diagnostic operationand a presentation of a graphical object for a user with myopic vision.Such processes may omit and/or add steps to the process 600. Similarly,steps of the process 600 may be performed in different orders than theexample order discussed above.

FIG. 7 depicts an example process 700 of a vision diagnostic operationand a presentation of a graphical output for a user with hyperopicvision. As discussed above, a user with hyperopic vision (e.g., afarsighted user) may clearly perceive faraway objects and may notclearly perceive nearby objects. With the use of proper correctiveeyewear (e.g., eyewear having a suitable prescription), the user may beable to clearly perceive both faraway and nearby objects. The process700 allows the user to perceive a graphical output in a manner bestsuited to the user's visual acuity.

At operation 702, the processor may perform a vision diagnostic processto identify a visual acuity of a user of the electronic device. Theuser's visual acuity may be determined by, for example, a visiondiagnostic test as described with respect to FIG. 3B. In someembodiments, a processor may receive a display adjustment request from auser generated input and may determine the user's visual acuity and/orvision condition based on the user's interaction with the electronicdevice. In some embodiments, the processor may determine the user'svisual acuity by a user input (e.g., entering the user's prescription ina setting, as depicted in FIG. 3A).

At operation 704, the processor may determine the user has hyperopiceyesight. The processor may determine that the user has hyperopiceyesight from the vision diagnostic process described with respect tooperation 702. In some embodiments, the processor may determine that theuser best perceives vision-corrected displays that have a hyperopicvision correction. In some embodiments, the processor may detect that a“Farsighted” slider is in the “ON” position.

At operation 706, the processor may direct an optical sensor to performa facial scan of a face of the user. Here, the processor directs anoptical sensor system (e.g., the optical sensor system 102) to perform afacial scan of the user. The facial scan may be performed in a manner asdiscussed with respect to FIG. 1 (e.g., by creating a three-dimensionaldepth map from a projected dot pattern). Once a scan of the user's faceis performed, the processor may determine that the scanned face shares afirst similarity score with a pre-registered identity depth map toconfirm an identity of the user. If the identity is confirmed, theprocessor may further determine if the scanned face corresponds to apre-registered alternate appearance, as discussed herein, by sharing asecond similarity score.

At operation 708, the processor may determine a corrective eyewearscenario of the user. For example, the processor may determine whetherthe user is, or is not, wearing corrective eyewear. The processor maymake this determination by comparing the facial scan captured atoperation 706 with a pre-registered identity profile including a set ofalternate appearances. For example, one alternate appearance maycorrespond to the user wearing glasses and another may correspond to auser with no corrective eyewear. The processor may determine whichappearance corresponds most closely with the scan taken at operation 706(by, for example, determining two similarity scores between the scan andthe two appearances and determining which similarity score is thehighest).

At operation 710, the processor may determine that the user is notwearing corrective eyewear, may generate a vision-corrected graphicaloutput, and may display the vision-corrected graphical output. Thevision-corrected graphical output may compensate for the hyperopicvision of the user by, for example, blurring a portion and/or theentirety of the standard graphical output; generating an overlay overthe standard graphical output; and/or making elements of the standardgraphical output larger, brighter, and/or more distinct. In someembodiments, the generated vision-corrected graphical output may replacethe previously displayed standard graphical output and may be presentedto the user instead of the standard graphical output. Additionally, thevision-corrected graphical output may only replace certain graphicalelements presented in the standard graphical output. Thevision-corrected graphical output may be a default graphical outputdesigned to compensate for a hyperopic vision or may be generated basedon the individual prescription of the user.

The vision-corrected graphical output generated at operation 710 may beconfigured specifically to compensate for a hyperopic vision deficiencyand may differ from other vision-corrected graphical outputs, asdescribed herein. For example, the vision-corrected graphical outputgenerated at operation 710 may be different from the vision-correctedgraphical output generated with respect to FIG. 6 at operation 612. As anon-limiting example, a vision-corrected graphical output generated tocompensate for a hyperopic vision deficiency while a user is not wearingcorrective eyewear may be significantly different than a standardgraphical output. Such a vision-corrected graphical output may, forexample, include elements that are larger, smaller, blurred, compressed,brighter, dimmer, color-shifted and/or stretched so that a hyperopicuser not wearing corrective eyewear may perceive the vision-correctedgraphical output as the user would when wearing the corrective eyewear(e.g., as the user perceives the standard graphical output as shown inoperation 712). The example above is merely explanatory and other typesof vision-corrected graphical outputs to compensate for a hyperopicvision deficiency may be used.

At operation 712, the processor may determine that the user is wearingcorrective eyewear and may display a standard graphical output. Such adetermination may be made in accordance with the scan taken at operation706 corresponding more closely to an appearance of the user where theuser is wearing eyewear.

In some embodiments, a vision-corrected graphical output may beinitially displayed to a user and the vision-corrected graphical outputmay continue being displayed at operation 710 and a standard graphicaloutput may be generated/modified and displayed at operation 712. In someembodiments, a standard graphical output may be initially displayed to auser and.the standard graphical output may continue being displayed atoperation 712 and a vision-corrected graphical output may begenerated/modified and displayed at operation 710.

The process 700 is an example process for a vision diagnostic operationand a presentation of a graphical object for a user with hyperopicvision. Such processes may omit and/or add steps to the process 700.Similarly, steps of the process 700 may be performed in different ordersthan the example order discussed above.

FIG. 8 depicts an example process 800 of an automatic vision diagnosticoperation and a control of a displayed graphical output. At operation802, the processor may direct an optical sensor system (e.g., theoptical sensor system 102) to capture dynamic facial data of a face of auser. In some embodiments, the dynamic facial data may be a video of theuser's face for a predetermined period of time. In some embodiments, thedynamic facial data may be a series of pictures taken at regularintervals. In some embodiments, the dynamic facial data may be a seriesof pictures of a dot pattern projected on the user's face.

At operation 804, the processor may analyze the captured dynamic facialdata to detect eye movements of the user while the user is perceivingthe graphical user interface on the electronic device. For example, theprocessor may use image processing techniques to detect a jitter ormovement of the user's eye. The movement of the user's eye may bemeasured over a predetermined period of time.

At operation 806, the processor may determine an eye strain condition ofthe user. The eye strain condition may correspond to or be based oncharacteristic eye shifting, squinting, or other type of eye movement.In some cases, the processor may determine a threshold likelihood thatthe user is struggling to view the standard graphical output based onthe detected eye movements (e.g., the processor may determine an eyestrain condition). For example, if the eye movements of the user arerapid and frequently dart back and forth, the processor may determinethat the user is experiencing eye strain. The processor may trackcertain visual fiducials on the user's eye (e.g., a center of the user'spupil) and may measure the movement of the fiducial to determine thethreshold likelihood that an eye strain condition is met. As usedherein, the eye strain condition may be used to refer to a variety ofpossible eye strain states of the user. The detected eye movement mayalso include a detection of a squinting or strain of the user's eyeduring a perceived reading activity. The processor may, through thedetection of eye movements, provide, for example, a numerical value of auser's eye strain condition. For example, if the eye strain condition isdetermined to be below a threshold value after a statistical analysis,the user may be determined to not be experiencing sufficient eyestrain.If the eye strain condition is above the threshold value, the processormay consider an eye strain threshold to be surpassed and may considerthe user to be experiencing heightened eye strain. At operation 810, theprocessor determines whether this threshold is met or surpassed.

At operation 810, the processor may determine that the thresholdlikelihood is not met or surpassed and that the user is not strugglingto view a graphical output. The processor then may direct an associateddisplay to display a standard graphical output.

At operation 812, the processor may determine that the thresholdlikelihood is met or surpassed, may generate a vision-correctedgraphical output, and may display the vision-corrected graphical output.The vision-corrected graphical output may be designed to reduce thevision strain of the user, for example, blurring a portion and/or theentirety of the standard graphical output; generating an overlay overthe standard graphical output; and/or making elements of the standardgraphical output larger, brighter, and/or more distinct. In someembodiments, the generated vision-corrected graphical output may replacethe previously displayed standard graphical output and may be presentedto the user instead of the standard graphical output. Additionally, thevision-corrected graphical output may only replace certain graphicalelements presented in the standard graphical output. Thevision-corrected graphical output may be a default graphical outputdesigned to compensate for a myopic vision or may be generated based onthe individual prescription of the user.

In some embodiments, a vision-corrected graphical output may beinitially displayed to a user and the vision-corrected graphical outputmay continue being displayed at operation 812 and a standard graphicaloutput may be generated/modified and displayed at operation 810. In someembodiments, a standard graphical output may be initially displayed to auser and.the standard graphical output may continue being displayed atoperation 812 and a vision-corrected graphical output may begenerated/modified and displayed at operation 810.

The process 800 is an example process for an automatic vision diagnosticoperation and a control of a displayed graphical output. Such processesmay omit and/or add steps to the process 800. Similarly, steps of theprocess 800 may be performed in different orders than the example orderdiscussed above.

FIG. 9 depicts an example process 900 of generating and displaying aprivacy screen in response to a facial scan of a user. For a standardgraphical output, a user may experience certain privacy concerns. Forexample, surrounding people may be able to view a display of anelectronic device in the possession of the user on, for example, acrowded restaurant or bus. If the user wanted to view highly sensitivecontent, the user would either need to move to a more private locationor physically block a view-line of the surrounding people. The process800 depicted here, creates a private graphical output that can only beperceived by a wearing of a particular set of glasses.

At operation 902, a processor of an electronic device may direct anoptical sensor system (e.g., the optical sensor system 102) to perform afacial scan of the user. The facial scan may be performed in a manner asdiscussed with respect to FIG. 1 (e.g., by creating a three-dimensionaldepth map from a projected dot pattern or by performing an imagerecognition analysis on a two-dimensional image). Once a scan of theuser's face is performed, the processor may determine that depth maps ofthe scanned face shares a first similarity score with a pre-registeredidentity depth map to confirm an identity of the user. If the identityis confirmed, the processor may further determine if the depth maps ofthe scanned face corresponds to a pre-registered alternate appearance,as discussed herein, by sharing a second similarity score.

At operation 904, the processor may detect the presence of a privacyeyewear on the face of the user from the facial scan taken at operation902. The privacy eyewear may be detected by comparing the depth mapstaken from the facial scan taken at operation 902 with previouslyregistered depth maps corresponding to an alternate appearance of theuser. The previously registered depth maps may have been marked as“Private” or may otherwise be listed as a private profile. In someembodiments, the privacy eyewear may be marked with a particulargraphic, QR code, bar code, and the like. The processor may detect thepresence of the marking and may determine the presence of the privacyeyewear. In some embodiments, the privacy eyewear may be provided as aseparate eyewear that intentionally distorts a user's vision. In someembodiments, the privacy eyewear may be standard eyewear owned by theuser.

At operation 906, the processor may perform a privacy blur operation tovary an appearance of a graphical output to be displayed. The privacyblur operation may be based on a distortion of the privacy eyewear. Insome embodiments, the privacy blur operation may blur in accordance withinformation stored, or associated with, a detected graphic, QR code, barcode, and the like. In some embodiments, the user may previously enterprescription information for the privacy eyewear and the privacy bluroperation may be based on such prescription information. At operation908, the processor may generate a blurred graphical output in accordancewith the privacy blur operation at operation 906.

Also at operation 908, the processor may display the blurred graphicaloutput. The blurred graphical output may compensate for the distortioncreated by the privacy eyewear vision of the user by, for example,blurring a portion and/or the entirety of a standard graphical output;generating an overlay over the standard graphical output; and/or makingelements of the standard graphical output larger, brighter, and/or moredistinct. In some embodiments, the blurred graphical output may onlyreplace certain graphical elements presented in the standard graphicaloutput. The blurred graphical output may be a default graphical outputdesigned to compensate for the privacy eyewear. The blurred graphicaloutput may also be referred to as a privacy blur, as presented herein,and may be a type of a vision-corrected graphical output, as usedherein. The blurred graphical output may appear unblurred when theprivacy eyewear is worn and may appear blurred when the privacy eyewearis not worn. In this way, a user wearing the privacy eyewear mayperceive the blurred graphical output with clarity while surroundingpeople not wearing the privacy eyewear may not perceive the blurredgraphical output clearly.

The process 900 is an example process for generating and displaying aprivacy screen. Such processes may omit and/or add steps to the process900. Similarly, steps of the process 900 may be performed in differentorders than the example order discussed above.

FIG. 10 depicts an example block diagram of an electronic device 1000that may perform the disclosed processes and methods. The electronicdevice 1000 may, in some cases, take the form of a mobile electronicdevice, such as a mobile phone; electronic watch; or laptop computer, ormay take the form of any other electronic device such as a television; acomputer display; or a display in an automobile. The electronic device1000 may be described with reference to any of FIGS. 1-9. The electronicdevice 1000 may include a processor (or processors) 1002, a memory (ormemories) 1004, an optical sensor (or optical sensors) 1006, ainput/output device (or input/output devices) 1008. Additional sensor(or sensors) 1010, a display (or displays) 1012), and a battery (orbatteries) 1014 may additionally be provided.

The processor 1002 may control some or all of the operations of theelectronic device 1000. The processor 1002 may communicate, eitherdirectly or indirectly, with some or all of the components of theelectronic device 1000. For example, a system bus or other communicationmechanism may provide communication between the processor 1002, thememory 1004, the optical sensor 1006, the input/output devices 1008, theadditional sensors 1010, the display 1012, and the battery 1014.

The processor 1002 may be implemented as any electronic device capableof processing, receiving, or transmitting data or instructions. Forexample, the processor 1002 may be a microprocessor, a centralprocessing unit (CPU), an application-specific integrated circuit(ASIC), a digital signal processor (DSP), or combinations of suchdevices. As described herein, the term “processor” may encompass asingle processor or processing unit, multiple processors, multipleprocessing units, or other suitably configured computing element orelements.

Components of the electronic device 1000 may be controlled by multipleprocessing units. For example, select components of the electronicdevice 1000 (e.g., the optical sensor 1006) may be controlled by a firstprocessing unit and other components (e.g. the additional sensors 1010)may be controlled by a second processing unit, where the first andsecond processing units may or may not be in communication with eachother. In some cases, the processor 1002 may determine a biologicalparameter of a user of the electronic device, such as a facialappearance, a biometric, and/or an eye strain.

The memory 1004 may store electronic data that can be used by theelectronic device 1000. For example, the memory 1004 may storeelectrical data or content such as, for example, audio and video files,documents and applications, device settings and user preferences, timingsignals, control signals, and data structures or databases. The memory1004 may be configured as any type of memory. By way of example only,the memory 1004 may be implemented as random access memory, read-onlymemory, Flash memory, removable memory, other types of storage elements,or combinations of such devices.

The optical sensor 1006 may detect image, video, and or opticalinformation from an environment surrounding the electronic device 1000.The optical sensor 1006 may be one or any number of individual camerasand may also include one or any number of light projectors. The opticalsensor 1006 may detect visible light, infrared light, ultraviolet light,or any combination thereof. In some embodiments, the optical sensor 1006may be a forward-facing camera to detect images in the same direction asa presented graphical user interface. Additionally or alternative, theoptical sensor 1006 may be disposed on the back of the electronic device1000.

The optical sensor 1006 may include a light projector which may projecta series of light beams onto an environment surrounding the electronicdevice 1000. The projected light beams may be of comprised of any typeof light including visible light, infrared light, and/or ultravioletlight. The projected light beams may additionally be detected by theoptical sensor 1006. The light projector may project a number of lightbeams so as to create a grid-type pattern.

The electronic device 1000 may also include one or more input/outputdevices 1008. In various embodiments, the input/output devices 1008 mayinclude any suitable components for detecting inputs. Examples ofinput/output devices 1008 include mechanical devices (e.g., crowns,switches, buttons, or keys), communication devices (e.g., wired orwireless communication devices), electroactive polymers (EAPs), straingauges, electrodes, some combination thereof, and so on. Eachinput/output device 1008 may be configured to detect one or moreparticular types of input and provide a signal (e.g., an input signal)corresponding to the detected input. The signal may be provided, forexample, to the processor 1002.

As discussed herein, in some cases, the input/output device 1008 includea touch sensor (e.g., a capacitive touch sensor) integrated with thedisplay 1012 to provide a touch-sensitive display. Similarly, in somecases, the input/output device(s) 1008 include a force sensor (e.g., acapacitive force sensor) integrated with the display 1012 to provide aforce-sensitive display.

The input/output device(s) 1008 may further include any suitablecomponents for providing outputs. Examples of such input/outputdevice(s) 1008 include audio output devices (e.g., speakers), visualoutput devices (e.g., lights or displays), tactile output devices (e.g.,haptic output devices), communication devices (e.g., wired or wirelesscommunication devices), some combination thereof, and so on. Eachinput/output device 1008 may be configured to receive one or moresignals (e.g., an output signal provided by the processor 1002) andprovide an output corresponding to the signal.

In some cases, input/output devices 1008 may be integrated as a singledevice or may be separate devices. For example, an input/output deviceor port can transmit electronic signals via a communications network,such as a wireless and/or wired network connection. Examples of wirelessand wired network connections include, but are not limited to, cellular,Wi-Fi, Bluetooth, IR, and Ethernet connections.

The processor 1002 may be operably coupled to the input/output devices1008. The processor 1002 may be adapted to exchange signals with theinput/output devices 1008. For example, the processor 1002 may receivean input signal from an input/output device 1008 that corresponds to aninput detected by the input/output device 1008. The processor 1002 mayinterpret the received input signal to determine whether to provideand/or change one or more outputs in response to the input signal. Theprocessor 1002 may then send an output signal to one or more of theinput/output devices 1008, to provide and/or change outputs asappropriate.

The electronic device 1000 may also include one or more additionalsensors 1010 positioned almost anywhere on the electronic device 1000.The additional sensor(s) 1010 may be configured to sense one or moretype of parameters, such as, but not limited to, pressure, light, touch,heat, movement, relative motion, biometric data (e.g., biologicalparameters), and so on. For example, the additional sensor(s) 1010 mayinclude a heat sensor, a position sensor, an additional light or opticalsensor, an accelerometer, a pressure transducer, a gyroscope, amagnetometer, a health monitoring sensor, and so on. Additionally, theone or more additional sensors 1010 may utilize any suitable sensingtechnology, including, but not limited to, capacitive, ultrasonic,resistive, optical, ultrasound, piezoelectric, and thermal sensingtechnology. In some examples, the additional sensors 1010 may includeone or more of the electrodes described herein (e.g., one or moreelectrodes on an exterior surface of a cover that forms part of anenclosure for the electronic device 1000 and/or an electrode on a crownbody, button, or other housing member of the electronic device 1000).

In various embodiments, the display 1012 may provide a graphical output,for example associated with an operating system, user interface, and/orapplications of the electronic device 1000. In some embodiments, thedisplay 1012 may include one or more sensors and is configured as atouch-sensitive (e.g., single-touch, multi-touch) and/or force-sensitivedisplay to receive inputs from a user. For example, the display 1012 maybe integrated with a touch sensor (e.g., a capacitive touch sensor)and/or a force sensor to provide a touch- and/or force-sensitivedisplay. The display 1012 may be operably coupled to the processing unit1002 of the electronic device 1000.

The display 1012 may be implemented with any suitable technology,including, but not limited to, liquid crystal display (LCD) technology,light emitting diode (LED) technology, organic light-emitting display(OLED) technology, organic electroluminescence (OEL) technology, oranother type of display technology. In some cases, the display 1012 maybe positioned beneath and viewable through a cover that forms at least aportion of an enclosure of the electronic device 1000. Many suchdisplays also include touch screen functionality where a user may exerta touch and/or a force on a touch-sensitive display to interact with anelectronic device via the display.

The battery 1014 may be implemented with any device capable of providingenergy to the electronic device 1000. The battery 1014 may be one ormore batteries or rechargeable batteries. Additionally or alternatively,the battery 1014 may be replaced or supplemented by a power connector orpower cord that connects the electronic device 1000 to another powersource, such as power transferred through a wall outlet.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to provide, forexample, facial recognition and/or eyesight diagnosis. The presentdisclosure contemplates that, in some instances, this gathered data mayinclude personal information data that uniquely identifies, may be usedto identify and/or authenticate, or can be used to contact or locate aspecific person. Such personal information data can include facialinformation, vision prescription information, demographic data,location-based data, telephone numbers, email addresses, twitter IDs,home addresses, data or records relating to a user's health or level offitness (e.g., vital signs measurements, medication information,exercise information), date of birth, or any other identifying orpersonal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the facial recognition data may be used to securean electronic device and may be used to generate and present avision-corrected graphical output. Further, other uses for personalinformation data that benefit the user are also contemplated by thepresent disclosure. For instance, eyesight prescription information maybe used to provide insights into a user's vision health, or may be usedto measure a user's vision over time to monitor changing eye conditions.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. In varioussituations considered by the disclosure, personal information data maybe entirely stored within a user device.

Personal information from users should be collected for legitimate andreasonable uses of the entity and not shared or sold outside of thoselegitimate uses. Further, such collection/sharing should occur afterreceiving the informed consent of the users. Additionally, such entitiesshould consider taking any needed steps for safeguarding and securingaccess to such personal information data and ensuring that others withaccess to the personal information data adhere to their privacy policiesand procedures. Further, such entities can subject themselves toevaluation by third parties to certify their adherence to widelyaccepted privacy policies and practices. In addition, policies andpractices should be adapted for the particular types of personalinformation data being collected and/or accessed and adapted toapplicable laws and standards, including jurisdiction-specificconsiderations. For instance, in the US, collection of or access tocertain health data, such as eyesight information, may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof facial recognition processes or eyesight diagnostic processes, thepresent technology can be configured to allow users to select to “optin” or “opt out” of participation in the collection of personalinformation data during registration for services or anytime thereafter.In addition to providing “opt in” and “opt out” options, the presentdisclosure contemplates providing notifications relating to the accessor use of personal information. For instance, a user may be notifiedupon downloading an app that their personal information data will beaccessed and then reminded again just before personal information datais accessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, determiningwhether a user is wearing glasses may be based on reflective propertiesof the glasses based on non-personal information data or a bare minimumamount of personal information, such as the content being requested bythe device associated with a user, other non-personal informationavailable to the electronic devices, or publicly available information.

The foregoing description, for purposes of explanation, uses specificnomenclature to provide a thorough understanding of the describedembodiments. However, it will be apparent to one skilled in the art thatthe specific details are not required in order to practice the describedembodiments. Thus, the foregoing descriptions of the specificembodiments described herein are presented for purposes of illustrationand description. They are not targeted to be exhaustive or to limit theembodiments to the precise forms disclosed. It will be apparent to oneof ordinary skill in the art that many modifications and variations arepossible in view of the above teachings.

As used herein, the phrase “at least one of” preceding a series ofitems, with the term “and” or “or” to separate any of the items,modifies the list as a whole, rather than each member of the list. Thephrase “at least one of” does not require selection of at least one ofeach item listed; rather, the phrase allows a meaning that includes at aminimum one of any of the items, and/or at a minimum one of anycombination of the items, and/or at a minimum one of each of the items.By way of example, the phrases “at least one of A, B, and C” or “atleast one of A, B, or C” each refer to only A, only B, or only C; anycombination of A, B, and C; and/or one or more of each of A, B, and C.Similarly, it may be appreciated that an order of elements presented fora conjunctive or disjunctive list provided herein should not beconstrued as limiting the disclosure to only that order provided.

What is claimed is:
 1. A method of controlling a vision-correctingoperation of a portable electronic device, the method comprising:scanning at least a portion of a face of a user using a sensor;generating a depth map using the scan conducted using the sensor;determining a similarity score between the depth map and one or morebiometric identity maps of a set of stored biometric identity maps thatare associated with a registered user; in response to the similarityscore exceeding a threshold, authenticating the user as the registereduser; determining a corrective eyewear scenario using the depth map;selecting a display profile that is associated with the correctiveeyewear scenario and the registered user; and generating a graphicaloutput in accordance with the selected display profile.
 2. The method ofclaim 1, wherein: the corrective eyewear scenario corresponds to theregistered user wearing a corrective eyewear; and the graphical outputcompensates for a vision deficiency associated with the correctiveeyewear scenario and the registered user.
 3. The method of claim 2,wherein: the depth map is a first depth map; the display profile is afirst display profile; the corrective eyewear scenario is a firstcorrective eyewear scenario; the graphical output is a first graphicaloutput; the method further comprises: scanning at least the portion ofthe face of the user using the sensor to generate a second depth map;determining a second corrective eyewear scenario using the second depthmap; selecting a second display profile that is associated with thesecond corrective eyewear scenario; and generating a second graphicaloutput in accordance with the selected second display profile; and thesecond corrective eyewear scenario corresponds to the registered usernot wearing the corrective eyewear.
 4. The method of claim 1, wherein:the threshold is a first threshold; the similarity score is a firstsimilarity score; and determining the corrective eyewear scenario usingthe depth map, comprises: identifying a subset of identity maps of theset of stored biometric identity maps, the subset of identity mapsassociated with the corrective eyewear scenario; and determining asecond similarity score between the depth map and the subset of identitymaps.
 5. The method of claim 1, wherein: the corrective eyewear scenariocorresponds to the registered user not wearing a corrective eyewear; andthe graphical output compensates for a vision deficiency while the useris not wearing the corrective eyewear.
 6. The method of claim 1, furthercomprising: detecting an eye movement of the user; and in accordancewith the eye movement corresponding to an eye strain condition,modifying the graphical output of the portable electronic device.
 7. Themethod of claim 1, wherein: the display profile is associated withprescription information related to a visual acuity of the user; and thegraphical output is generated based, at least in part, on theprescription information.
 8. An electronic device comprising: a housing;a display positioned at least partially within the housing andconfigured to display a graphical output; a transparent cover positionedat least partially over the display; an optical sensor positioned belowthe transparent cover and configured to obtain a scan of a portion of aface of a user; and a processor configured to: generate a depth mapusing the scan; determine a similarity score between the depth map andone or more identity maps of a set of stored biometric identity mapsthat are associated with a registered user; in response to thesimilarity score exceeding a threshold, identify the user as theregistered user; determine a corrective eyewear scenario using the depthmap; select a display profile that is associated with the correctiveeyewear scenario; and generate a graphical output in accordance with theselected display profile.
 9. The electronic device of claim 8, wherein:the optical sensor comprises a light emitting module configured toproject a dot pattern on the portion of the face of the user; and theoptical sensor obtains the scan of the portion of the face of the userusing the projected dot pattern.
 10. The electronic device of claim 9,wherein: the projected dot pattern is produced by a series of infraredlight rays emitted from the light emitting module toward the portion ofthe face of the user; and the optical sensor further comprises aninfrared-sensing array configured to detect infrared light reflectedfrom the portion of the face of the user.
 11. The electronic device ofclaim 8, wherein the corrective eyewear scenario corresponds to theregistered user wearing a corrective eyewear.
 12. The electronic deviceof claim 8, wherein the corrective eyewear scenario corresponds to theregistered user not wearing a corrective eyewear.
 13. The electronicdevice of claim 8, wherein: the corrective eyewear scenario correspondsto the registered user wearing a privacy eyewear; and the graphicaloutput includes a privacy blur that appears unblurred when viewed usingthe privacy eyewear.